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An examination of postmodern grounded theory and narrative research methodology 
philosophies should make us rethink foundational issues in writing assessment design. 
Postmodern approaches emphasize interpretation and meaning-making, arguing that truths are 
made rather than found. In fact, many postmodern researchers emphasize the impossibility of 
objective, reliable results, especially in human research. In a communicative field such as 
teaching writing, this makes particular sense—the myriad of factors involved in a student writer's 
progress and in a teacher’s ability to facilitate writing development cannot be isolated from each 
other. 

As Jeff notes, the Art Institute of Pittsburgh was working in the direction of qualitative 
assessment even before I had been hired. I helped word our reasons for qualitative assessment 
based on my experiences with qualitative research, but my predecessor April Sikorski was my 
inspiration, as was Jeff in his explanations of his previous departmental professional 
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development experiences, which I noted could be reworked a tad to create a qualitative research 
format. This led me to ask: is assessment research? What is changed by looking at assessment as 
research? I must answer these questions here briefly before I continue discussing our postmodern 
turn. 

Assessment as Research? 

According to the Oxford English Dictionary, the fifth definition of assessment is “estimation, 
evaluation” with a sub-definition of “the process or means of evaluating academic work; an 
examination or test” (OED online). Definitions one through four, which are preoccupied with 
numbers, perhaps influence definition 5b, which describes educational assessment, as 
educational assessment has a long history of being preoccupied with numeric representations of 
evaluation. 

There is a whole field of writing assessment that is rich with history and methodological 
developments put forward by passionate and brilliant compositionists as outlined by Kathleen 
Blake Yancey and by Brian Huot and Peggy O’Neill. As I read about assessment, I saw 
correlations between assessment work, “the process or means of evaluating” academic progress, 
and the reading I had done on research methodologies; for example, both fields included 
discussions about criteria of evaluation, validity, and what questions were important to ask. 
However, that does not make research and assessment the same thing. 

According to the Oxford English Dictionary, a definition of research is “a search or 
investigation directed to the discovery of some fact by careful consideration or study of a 
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subject” (OED online). The first definition of research, then, involves searching carefully for a 
person or thing. So while assessment deals with evaluation, research has us investigating toward 
a discovery. Those are two very different goals. Evaluation seeks definite answers, like whether 
certain goals are being met. Investigation more broadly tries to see what is happening in a 
situation. Still, adding the word qualitative to assessment changes the game because it adds to the 
mix the questions of why and how. As Shaun R. Harper, editor of the New Directions in 
Institutional Research special issue on qualitative assessment, elaborates: 

qualitative techniques can uncover the hidden forces that drive behaviors, shape 
experiences, and influence outcomes on college campuses. Finally, qualitative assessment 
methods can illuminate the voices of various groups and individuals throughout 
institutions of higher education. One of the key advantages of qualitative research is its 
ability to allow participants to construct meanings and insightful interpretations of their 
own experiences. (3) 

Note how in this quote, when referring to qualitative assessment, the words research and 
assessment seem interchangeable. This is largely because qualitative assessment involves both 
evaluation and investigation. Rubric assessment, while it has qualitative descriptions, primarily 
answers narrowly defined questions. It does not answer the questions why or how, so much as it 
answers what happens, like quantitative assessment, and that is when rubric assessment works 
well. This is one reason why the AiP English department struggled with it. Qualitative 
assessment would by definition answer more than the question of whether certain goals are met. 
It would investigate why goals are met or not. Harper and George D. Kuh argue that qualitative 


89 



THE CEA FORUM 


Summer/Fall 

2010 


assessment “can help answer some of the complex, vexing questions that concern various 
stakeholders in higher education” (6). In order to answer a complex question like why, 
assessment would need to incorporate a research methodology that investigates more generally 
what happened in a course. Thus, assessment and research are merged. An evaluation of how 
well a course works across a department is conducted as research, and the investigation’s 
documentation functions as an assessment of a course. In fact, Bob Broad’s study of writing 
assessment shows how qualitative investigations can not only evaluate a program’s work but also 
provide the researchers, the subjects, and the stakeholders with a better understanding of 
strengths and weaknesses in a program. 

Realizing that qualitative assessment was an investigation and an evaluation led me to the 
conclusion that in order to do qualitative assessment well, we needed to treat it as a qualitative 
research project. I warned the faculty that in my experience, qualitative research required a lot of 
work, from finding and defining a theoretical stance, to developing a methodology, to conducting 
the research, and then analysis and writing. Together, we began to read various qualitative 
research books, including the four volume set The American Tradition in Qualitative Research, 
edited by Nonnan K. Denzin and Yvonna S. Lincoln. 

Choosing a Theoretical Framework 

As we refined the theoretical stance for our research, the faculty found themselves drawn to 
postmodernism, especially postmodern grounded theory as described by Adele Clarke in her 
book, Situational Analysis: Grounded Theory after the Postmodern Turn. We liked grounded 
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theory, for as Katie notes, we were only beginning to understand ourselves as a department and 
we barely knew what questions to ask. We liked Adele Clarke’s movement of grounded theory 
around the postmodern turn because it emphasizes the importance of open questions, complex 
and uncertain answers, and contextuality. Furthennore, it embraces the continuity of researcher 
and researched. Postmodern grounded theory pushes against simplification and emphasizes an 
understanding and acceptance of complexity and it encourages multiple voices and perspectives. 
We were intrigued by how Clarke’s theory strove to not be conclusive, but instead “tentative, 
open, jarring, troubling” (32). Arguments among the group ensued over this—didn’t we want to 
be conclusive? What was lost by conclusiveness? Ultimately, we decided that it was the 
conclusiveness of rubrics that troubled us and that we were open to trying more tentative 
contextual answers as it was more honest. 

In fact, it started to become apparent that another key part of our faculty’s dissatisfaction 
with the previous assessment method occurred because the core beliefs of most of the faculty, 
whether consciously or not, were either rooted in a postmodern mindset or conflicted. Either 
way, searching for codified answers proved unsatisfying and antithetical to teachers who used 
practices that valued creativity and subjectivity, as is discussed by Katie and Karen. As Katie 
reveals, postmodern theories made a lot of sense to us as writing teachers, especially, perhaps, 
because we worked in an Art Institute. Art is contextual, open to interpretation, and lacks 
absolute conclusive strictures. In art and writing, rules can be bent or broken, provided the 
correct application and context. This was something our rubrics could not sufficiently address. It 
makes sense that teachers who work with artists and have observed changes in fashion and social 
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expression can appreciate postmodern thought and agree with Shirley K. Rose and Margaret J. 
Finders when they say that postmodern education “acknowledges the instability, provisionality, 
and contingent nature of what constitutes good teaching practices [.. .and ] that seeks not closure 
and definition but a commitment to keep thinking together” (85). We agreed that assessment of 
our courses should do the same. An evaluation of a postmodern course should provide a record 
that acknowledges and reflects such openness and collaborative searching for a complex and 
nuanced understanding of departmental practices. 

Our Methodology 

A postmodern qualitative approach could allow for a co-created and co-conducted research 
methodology, where we were simultaneously researchers and the researched (Kuhn and Woog; 
Nora), so we chose to study ourselves as a focus group and to together discern how and why 
students earned the grades they did in our classes. We asked the question “What does an A, C, or 
failing paper and portfolio look like in my class?” and provided sample portfolios from our Fall 
2007 term to show as examples. We had the quantitative data of our students’ grades that were 
recorded each term already, but we wanted to understand how each instructor graded and the 
reasoning for that grading process. Thus, we would better understand if departmental objectives 
were being met, and if not, why? We were not only finding out if an instructor focused on certain 
objectives more than others, and why, but giving each other the benefit of the doubt that there 
perhaps was a reason if, for example, Standard English (one of the course objectives) was not 
being taught and assessed. 


92 



THE CEA FORUM 


Summer/Fall 

2010 


The conversations we had about our portfolios were recorded and transcribed so that we 
could later refer to them as a record of our discussions about how and why we grade as we do, 
including all the messy blurry issues that would be missed in a summary. Before deciding on an 
analysis method, we read through the transcripts once and then met to discuss them. We agreed 
that we saw key themes surfacing, so we chose to use an analytical narrative research 
methodology, categorical content analysis, as described by Amia Lieblich, Rivka Tuval- 
Mashiach, and Tamar Zilbur in their book Narrative Research: Reading, Analysis, and 
Interpretation. We chose this because we noted categories of discussion that emerged as we 
talked about the portfolios, and we wanted to be able to closely examine our discussions and 
assumptions in each category. Especially important was to ascertain what were the most 
important questions that arose so that we could continue to come back to them in future research/ 
assessment. We did not limit ourselves to our original question because we sought, “not closure 
and definition but a commitment to keep thinking together.” So we coded the transcripts on our 
own to come up with general categories, then collectively named the categories: 

1. the nature of English 095 

2. evaluation of English 095 papers 

3. the composition course sequence 

4. student and instructor issues 

5. tensions between discourses 

6. language issues in writing 

7. writing across the curriculum 
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We also noted an eighth category, which marked self-reflective moments where we 
discussed, in the midst of the assessment, how the qualitative assessment was changing how we 
viewed ourselves, our department, and how we grade. In total we did three levels of coding. We 
coded the transcripts again according to the eight categories, adding in a multitude of 
subcategories. Then we checked each other’s coding for each individual category. We noted as 
we worked that not only was qualitative assessment going to provide us with answers about how 
and why we graded as we did, it was also changing the future answers to those questions as it 
was being conducted. Finally, we wrote a report of our assessment, explaining the theory and 
methodology, and analyzing the results in each category, which ended up being about fifty pages 
long. 

One of the benefits of our assessment was that we could no longer hide behind pseudo¬ 
anonymity. Each faculty member was responsible and accountable for how he or she graded 
assignments, for defending his or her theories that led to certain fonns of grades, and for 
explaining exceptions. All of this was done on public record, transcripts of which would later be 
analyzed and possibly quoted. These transcripts, as well as the copies of the portfolios and the 
analytical report, would be kept for future reference and possible research. These materials made 
the departmental administrators aware enough of instructors’ individual assessment practices to 
be able to explain, and be accountable for, how each course and the courses as a group were 
taught and assessed. Accountability was localized and contextualized. However, the picture of 
the departmental assessment practices for English 095 in Fall 2007 could only be provisional and 
contingent upon the individual instructors teaching that semester. We took our cue from Shirley 
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K. Rose and Margaret J. Finders, integrating their explanation of postmodern education with our 
explanation of our assessment: through evolving methodologies designed to suit the specific 
place and the specific time, postmodern qualitative assessment can provide a rich, descriptive 
record of how a department continually evolves in its acknowledgement of the instability, 
provisionality, and contingent nature of what constitutes good teaching practices, seeking not 
closure and definition but a commitment to keep thinking together. 
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